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PROFESSIONAL SCHOLARSHIP IN EDUCATIONAL TECHNOLOGY; 
CRITERIA FOR JUDGING IflQUIRY 

I n tro dtictl on 

As a professional field such as aducatlonal technologj alternates 
between evolution and revolution; it is, occasionanyi necessarj^ to re- 
examine some of the basic assumptions which guide its development. This 
article, commissioned by ERIC and AV Cofnmuni cation Review^ (AVCR), is 
directed to the publication criteria which are used tD judge scnolarly 
activity in educational technclogy. 

In recent years with the advent of more specialized roles In 
Educat-'onal Technology as well as Education at large there appears to be 
some confusion over publication standards that are appropriate for a 
scholarly journal such as AVCR. In recent correspondence with the 
Division for Instructional Development^ of AECT the author has found a 
wide range of opinions regarding what should constitute the nature of 
research, development, and what has constituted the nature of previous 
publication policies. The responses ranged from requests for normative 
data based survGys of media utilization to the observation that the practi< 
of development was not a suitable topic for "the only American journal to 
focus exclusively on research in our field," 

Soiiie of the confusion regarding the concepts of research, devel opnien t , 
and practice seems to be a matter of how fnany functions one argues^ are 
being performed. Baker (1973) reviewed the various positions regarding 
research and development that have been taken in recent years. The review 
showed some have argued that conclusion inquiry and decision oriented 
inquiry subsume the distinction between research and development. Others 
have argued that the process of information flow Is the most salient 

*The author informally polled the approximately 400 members for their 
opinions regarding the problems of publishing in conjunction with their 
Instr-ucti onal Deveiopment activities. Unfortunately, space does not perniit 
the opportunity to thank each of the respondents. 
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discriminating factor. For eKaniplei linear, i nteractlona 1 i st, and cyclic 
forniula tions have been designed to model the Information rela ti onshlps 
between researchs dcvelopnients and practice. 

The position taken here is that the classic concepts of roles and 
functions can be applied to distinguish research, development^ and practice* 
Adapting the work of Gideonse (1963) it can be argued that there are three 
professional roles being performed, These roles can be characterized by 
different expectations, contents, and particularly outcomes. 1) Practi- 
tioners in educational institutions are expected to deal with the buzzing* 
blooming confusion of the real world and control the variables that lead 
to knowledge, skills, attitudes-, etc., in their students* 2) Educa t1 onal 
res earc hers ^ on the other hands deal with the confusion of uncertain 
theoretical understandings and move toward the discovery of new knowledge- 
This knowledge is appropriately tied to educational problems^ but the 
outcomes are valued for their reduction of uncertainty^ not necessarily 
for their inimediate utility. 3) Devel oper s (including many education 
technologists) occupy a middle ground. They are constrained by Immediate 
problernSs but, unlike practitioners^ their efforts are more directed by 
scientific repllcable processes which result in pragmatic outcomes. On 
the other hand^ unUke researchers^ the developers' most Important 
criterion for success is the immediate utility of the processes or products 
they invent. 

The central argument of this article is that underlying the professional 
roles is one function: that function is inquiry.* The point is not to 
make the definition of inquiry so broad that it includes all the activities 
In all professional roles^ but to assert that inquiry is the single function 
that integrates all three roles. Inquiry is seen as a formal function that 

*The term inquiry Is utilized here because it appears to enjoy a broader 
and more inclusive ineaning than 'research,* Also, It helps to clarify the 
^distinction bet^reen the role of research and the function of inquiry, 
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may take different fof^ms with different goals, constraints, role percep- 
tions, etc.; hov.'tjver, tihe current definitions and commonly held criteria 
for judging inquiry are inclusive and applicable to inquiry associated 
with any of the professional roles educational technologists assume. If 
accoiTiniodati 0 IS for role in publishing are to be made they may be directed 
to more preiise coimnu n i cat ion between professionals who serve in editorial 
functions pnd those who are engaged ^'n the various roles. 

S Q me J3e^ f i n i_t j n n^s^ 

Educational Ing^u^ir.v is defined as a process which includes problem 
formation, hypothesis fcrniation, and hypothesis verification. Further, 
the process is restricted to the domain of planning, implementing, and 
evaluation of the ma nagemenc- of- 1 earn 1 ng process. This definition has been 
adapted from E. Steiner Macda ( 1 976) and Hoban ( 1 965). T^ie classical 
processes of inquiry have been utilized in a definition by Maccia, In 
regard to domain, Hoban has argued that the management of learning problem 
subsumes the so-called teach i ng- 1 earn i ng process, the term utilizeH by 
Naccia , 

The classic proress distinctioris have formal counterparts across many 
disciplines or fields of study. For example, the definition of a problem 
can occur in a symbolic fflathematical form or in a Fruedian expression 
describing the psychic disability of a child, etc. In all cases the 
problem sets out a question which a professional comniunity of scholars 
has not resolved. The hypothesis formation process is a process wherein 
tentative solutions to the problems are posed in a manner consistent with 
the modus qperandj^ of the discipline or profession. The hypothesis 
verification process includes the formal disciplined resolution of the 
problem in a manner that enables other scholars to incorporate that 
solution in subsequent explorations of the same or similar problems. 
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The restriction in domain is a characteristic of a profession, In 
the same manner that medicine draws on the 'mother' disciplines of chem- 
istry, biology, etc., to create a form of knowledge peculiar to its 
sphere of professional interests, education, also, draws on many discip- 
lines and specific methodologies to create a unique body of knov^ledge 
that bears on educational probleins. The terminology in this definition 
was deliberately chosen to incorporate philosophic as well as empirical 
i nq u i ry . 

The goals of educational inquiry are understanding and prediction 
(precision)*. This almost axiomatic definition has been restated by 
Dubin (1976) for an applied field. Following Bergman (1957) he argues 
that understanding results from a knowledge of the process or dynamics of 
a theory. Prediction results from investigation of the outcomes of a 
theory. The distinction is in part a matter of the level of description 
one uses. In inquiry directed toward understanding the professional 
focuses on the abstract relationships or generalizations that constitute 
a logical explanation of the reality being explained. In Inquiry directed 
toward prediction of a professional focuses on the less abstract empirical 
laws which have been derived from the theory. The empirical laws are 
applied when the measures of one set of theoretical variables are utilized 
to predict the measures of another set. Dubin argues that these dual goals 
are evident in theory building that is 'applied' or 'basic' He also 
argues these two goals are coordinate in that knowledge cannot be advanced 
without both goals being pursued. 

Argyis (1976) has argued that professionals In an applied field 
often exhibit a predisposition toward the goal of prediction in inquiry 
with the consequent stagnation of understanding of theoretical processes. 

*The terms prediction and precision will be considered equivalent for 
the purposes of this discussion. Precision is more useful in contexts 
such as discussions of inquiry directed to improving professional practice. 
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In other words, a theory that permits a reasonable level of prediction 
is not brought under exannnation. For example, it could be argued that 
the use of individual differences theories for prediction of academic 
success have enjoyed 'sufficient' success thereby curtailing the 
development of new theoretical formulations of individual differences 
and school learning. In essence, Argyis argued that the constraints of 
professional roles act to preclude the advance of knowledge, Argyis* ' 
observation appears to be usefi^l in evaluating inquiry in educational 
technology, as well. For exaniple^ understanding of the mastery model 
has not substantially improved since the original formulations of 
Carroll (1963) and Bloom (1968). Subsequent exploration has been largely 
oriented toward improving the precision of the methodology (e*g,, Block, 
1971 ) . 

Inquiry in Educational Technologv is defined as all educational 
inquiry, associated with the planning, implementing, and or evaluation of 
the management of learning process, which employs a systematic technolog- 
ical process. The significant addition to the previous definitions is the 
phrase, systematic technological process. Following authors such as 
Finn (i960), Hcban (1965), and Heinich (1970) it is argued that an 
appropriate definition of educational technology must include reference 
to a systematic process. That is, tecnnology defined with hardware or 
software attributes Is too transitory to build useful philosophic 
dlsti^nctions and^ therefore, too restrictive. Many current definitions 
of technological processes may be found in development models (e.g., Davies 
& Schwen, 1972) or message design models (e.g*, Fleming & Levie, in press). 
These models all have the common attributes of, a) disciplined analysis ^: 
problem^ context, constraints, learners and task; and, b) disciplined 
synthesis involving: the design of replicable forms of instruction and 
formative and summative evaluation. It is true that professional practice 



quite often involves the use of 'media' hardware and software. This 
author would argue that a formal definition of educational technology may 
not be bound by attributes of hardware and software. 

In qui ry iji Ins true t Ion a 1 Devel opmen t and Inquiry in Educational 
Technology will be treated as coordinate terms for the purposes of this 
discussion* If the reader were to include media attributes in the 
definition of Inquiry In Educational Technology, Inquiry in Instructional 
Development would bo the superordi na te term because Inouiry that incorpor- 
ated a technological process without media hardware or software would 
constitute a more c oniprehens i ve logical set, 

Implica tions Fonov/ing From the Defi nitions 

An important assumption held in respect to this article is that AVCR 
should publish inquiry that advances the profession of educational 
tec hnol ogy * 

The definitions allow for a middle ground between those professionals 
who would restrict publicatinn in the professional Journal to 'empirical' 
inquiry directed to theoretical understandings and those professionals 
who would seek sophisticated reports of development or practitioner role 
activities. To restrict publication to empirical studies directed toward 
understanding or prediction is to operate with a definition of Inquiry 
that is too narrow or restrictive. For examples there are many questions 
of philosophy that professionals in educational technology will need to 
address. A reasonable outlet for that type of inquiry is the professional 
journal. On the other hand, to publish sophisticated development or 
practitioner role activities as inquiry is to be unduly repetitive 
regarding commonly understood generalizations and facts or commonly utilized 
methodol ogies , 
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The use of the rather broad definition of Inquiry in Educational 
Technology opens the journal to a broader range of topics than the current 
editorial policies would seem to permit. For exaniple, an educational tech- 
nologist/developer could develop a so-called auto-tutorial class. Associated 
with the systematic technological analysis used in the design of the class; 
a management system employing student tutors could be utilized for remedial 
purposes. Inquiry which explored the effectiveness of various tutorial 
procedures, even 1f the procedures excluded 'media' developed for the class, 
would be an appropriate topic for publication under the proposed definition. 
Of course, the example could be expanded by excluding the auto- tutorl a 1 
aspect. The class could be traditional in the sense that media would not 
be used. By the definition, if a systematic technological process were 
employed, inquiry could be conducted that would not exceed the logical 
reach of the definition. 

The Crite r i a f or _J_u.d ging Scholarly P r o d u c t s 

It IS not sufficient to define inquiry and expect reasonable publica- 
tion practices and procedures to be deduced. In this section, some of 
the classical criteria which characterize adequate inquiry will be reviewed. 
These criteria will be submitted as a partial list to be debated, expanded, 
or modified by those who Judge the adequacy of scholarly endeavors in our 
profession. This restatement of classic criteria is intended as a 
stimulant toward the re-exaniina tion of basic assumptions alluded to in the 
opening paragraph of this article. 

I - Inquiry in Educational Technology shou ld be p ubl i c ly ver 1 f lab 1 e . 
Professionals in educational technology have the obligation that all 

*The author wishes to acknowledge his colleagues on the Commission on 
Doctoral Study, School of Education* Indiana University* particularly 
A* Stafford Clayton, who led the committee in developing an earlier 
version of these criteria* Those criteria appeared in a report entitledi 
Report of the Commission of Doctoral Study (undated 1975). 

ERiC 9 



o 



professionals do to describe their scholarly activities in a manner which 
permits the referent community of scholars to: 1) examine the basic plan 
or design that has been used; 2) replicate the processes that have been 
employed; and, 3) accept or reject the outcomes that have been developed. 
Empirical and hi s tor i ograph i c inquiry may be particularly enhanced by 
the media often used in this profession. An unusual opportunity to 
preserve the data or treatments exists. However, it would appear that our 
documentation procedures are lacking. For example, most empirical studies 
are impossible to replicate because mediated treatments are unavailable. 
An archival library is needed for the field. An Interim solution has been 
developed by Merrill ( 1 965 ). He provided access to the treatinents in his 
several sequencing studies by placing the original work on microfiche with 
a national documentation center. 

II. Inquiry in Educational Technology should be disciplined. Disciplln 
as a criterion is advanced In place of and in contrast to the often stated 
concept, systematic inquiry. Cronbach and Suppes (1969) made the point 
when they suggested that inquiry in practice is not always ordered or 
systematic, either in regard to procedures or in the creaf^'on of a new 
subdiscipllne or field. The quality that does characterize excellence in 
inquiry is the style of the investigation. Inquiry is disciplined in the 
sense that: the problem is described with precision; the assumptions and 
the boundaries of the inquiry are carefully delineated; the report of the 
design or plan is explicitly focused on logical consistent outcomes; and 
the outcomes are set forth in language or syntax that is descriptive, 
coherent and concise. It is unfortunate that the criterion has been mis- 
applied to the preliminary procedures of inquiry, since for this reason 
students of the profession as well as in other fields seem to carry a 
perception of inquiry that is formal, empirical, rational and mechanistic. 

10 
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The rich, speculative, creative, frustrating and insightful early stages 
of most inquiry is, perhaps, distorted by the systematic formality of the 
final report. It may be appropriate to speculate that this perception of 
the inquirer has been responsible for forming the collective pattern that 
many authors decry, e.g.. Snow and Clark (1976), Levie & Dickie (1973), 
etc. Departing from this narrow model of quantitative empirical inquiry 
could conceivably advance the field as fnuch as increased sophistication- 
of methodology. Of course, logical, historic and philosophic forms of 
inquiry should be encouraged, not for the sake of expanding the forms, 
but to facilitate integrated advances in knowledge. 

11 K Inquiry 1 n Educ a ti ona 1 Technology shQuld be directed toward 
general izabi 1 i ty , The concept is one of intent rather than absolute state. 
At a general level, this concept is not advanced with the premise that all 
inquiry will discover new or novel rules, principles, or facts^ but with 
tlie expectation that deviations from commonly understood generalizations 
or understandings are to be carefully reported. In other words, the 
complexities of inquiry generally preclude the uncovering of substantial 
new knowledge, whether in Just one or 1n a number of studies. General- 
izability of knowledge may be achieved through continuous effort across 
a series of studies; therefore, individual reports must be evaluated In a 
well defined context. 

On a technical level, there have been a number of advancements in the 
concept of validity that should have substantial influence on inquiry in 
our field. CampbelVs and Stanley's (1 966 ) well-known contribution on the 
methodology of quasi-experimental design and on internal and external 
validity has been augmented by more recent contributions such as Snow (1974), 
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Cook and Campben (1976). nnd the most recent AVCR-ERIC commissioned 
paper by Snow and Clark (1975), which reviews inquiry in our field. 
Including decision theory within this context is especially valuable for 
those professionals who are primarily concerned with precision. For 
example, Schwen (1973) has argued that the application of decision theory 
constitutes a logic 1 extension of the formal analysis function in 
instructional development. It is a means of using empirical data to 
logically analyze the consequences of extending a technological treatment 
to other samplGS or individuals. Decision theory, therefore, constitutes 
a useful extension of the external validity concept in the prediction of 
behavior, 

I V . Inquiry in Educational Technol ogy s ho u 1 d be incorporated with a 
c^oncjeptua^l_^s tr uctu re o r theoretical f ra mewor k . Excellent inquiry cannot 
occur in an intellectual void. All scientific advances, even revolutionary 
advances (Kuhn, 1970), are based on the accumulation of past theoretical 
understandings. In an applied field as well as in a "classic discipline," 
a quality of excellent inquiry 1s that it focuses on theoretical or 
conceptual problems^ and not exclusively on day to day tasks or symptomatic 
indices of fundamental problems. The predisposition of a professional 
field such as educational technology to engage inquiry directed toward 
prediction makes this criterion all the more relevant. Educational tech- 
nologists must continually return to the field's cumulative knowledge, 
as well as to the disciplines, to define professional problems in increasing 
ly precise theoretical terms. Ultimately, new theoretical understandings 
as well as increased precision will be a product of this willingriess to 
question traditional definitions of problems* 

V * Inquiry in Educational Technology should be directed toward an 
extensi on of know 1 edge , As noted above, additions to knowledge are seldom 
dramatic. New knowledge is seldom clearly disparate from previous 
jnderstandings. In most instances of inquiry it would be optlniistic to 
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e than a subtle refinement of current knowledge C7^a definition 
ela ti Otis fii p between previously discontinuous categories of 

Also, as Helnich (1S70) and Macda (197 6) hav^e observed , the 
educational inqulrj are. unique to ed uca t iona 1 pr obi ems . That 
owl edge gained from educational inquiry 1 s mo re than an extension 
ogy orother discipllres. It follows tha t ac cumu lated knowledge 
onal technology Is discrete from tde 'mother' disciplines, 
the problem definition may be influenced by the classical 
of scholarship, however, the definition of t lie problem and 
ed extension of knowledge niust be def ended fr cm t he context of 
al problems. 

La^yjjLLJJL ^toAtlorj 1 Techn olo gx shou Id demo rstr ate a comprehensive 
ie definition of a specific ins truinen tal problem «f practice Isj 
^ ilnsu ffici ent to merit its consideration by a co mmun1 ty of 
The 1 nstrumental problem is and shoul d be a representation of 
iresolved logical problem. The cl ass 1 c s e arc ii of the literature 
ir of functions, the most crucial of which is to relate the 
)roblen to the larger cont ex t o-f educa tion al issues which it 
i. The inquiry must ultimately stand on the meri -ts of this 
logical problem. Mo amount of methodological or logical 
:ion can compensate for an 111-deflned pro blem ststement or a 
iloped contextual formul at 1on o-f th e p robl em. In educational 

as in all professional fields, th e u rres ol ved 1 ogical problem 
ated to professional problenis in a meaningful fashion, 
a problem that attempts to speak t o a n ex tremely broad set of 
es or instances such as the deyelopment of a theory of learning 
tter left to the ap pr oprl ate d 1 scl pi 1 ne. On the other hand, a 
ch is wholly 1 nstruine ntal or pr agmati c ha s been t;oo narrowly 
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defined. A properly defined problefn stands on both the common pragmatic 
foundation of professionai experience in the field, and the unique set of 
accumulated knowledge, 

VII. Inquiry in Educational TechnoVog^ s hould dernonstrate creative 
exploration. In many instances a "discovery" 1n inquiry is a reordering 
or reformulation of understandings. In these cases the "discovery" may 
create a different perspective or insight regarding theoretical under- 
standings, or regarding precision in practice^ or both. The question is 
not whether the discovery ircorporates new or novel elefnents. A 
contribution is measured by the potential effect on the professional 
community, A creative ''discovery" modifies the previous definition of 
the related Rducatlonal problem in a manner that must be considered in 
future inquiry, 

VIII, Inquiry in Educational Technology m ust make sense. This 
criterion Is offered as an extension of the Vers teken concept popularized 
by Winch (1958) and others (Gibson, I960), This concept was originally 
discussed in the context of social theories as a means of distinguishing 
between inquiry which revealed a "sympathetic understanding" of the 
phenomena being investigated, and Inquiry which treated problems 
mechanistically, The sympathetic understanding was deemed necessary to 
develop the subtles humane understandings that were fundamental to the 
problems being eMploreds such as a sociological description of gang war- 
fare, V erst eke n was to be achieved by an inquirer who lived as a 
participant in the events. An inquirer who merely relied on observations 

as an "outsider,'' was thought to produce sterile or insensitive understandings 
of the phenomena. This criterion seems relevant to an applied field such 
as Educational Technology as well. It's not a matter of all professionals 
'living in' the practical problems^ although, that could be appropriate 

o 14 
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training for a career as a researcher. What is required for sensitive and 
intelligent description of educational problems is an involvement or 
participation in the maiiagement of learning process that forces the 
inquirer to periodicany attend to the assumptions, the analytic perspectiv 
and the methodologies that have altered or destroyed the reality being 
modeled. 



Insert Tabl e 1 
about here 



Impl 1 ca t i on s 

These criteria have been developed with the explicit intention that 
they could be used to judge inquiry in Educational Technology. However, 
there are two Important qualifications: first, not all inquiry suitable 
for consideration by educational technologists will meet all of the 
criteria. These criteria are intended to serve as an Ideal. It is quite 
possible that excenence in one aspect of inqtiiry will need to be weighed 
against an adequate or questionable quality of another aspect. These are 
the decisions that editorial boards must weigh. Prospective authors have 
the responsibility to make explicit the strengths as well as weaknesses 
1n the processes and products of their inquiry. Second, the criteria are 
not exhaustive, further clarification and modification of the criteria and 
consequent editorial policy would be expected. The criteria cut across a 
number of different philosophic Issues. It is Inevitable that the 
positions taken here will need to be modified by subsequent inquiry. 

To further illustrate the utility of the criteria a series of exemplars 
and false exeniplars have been created to facilitate use in editorial 
decision making, 

15 
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Extensive descriptive evaluations documenting the installation of a 
technological Innovatiohi or development project, etc.* are often false 
exemplars. Evaluations often are verifiablei disciplined ^ creative^ and 
sensible. However^ they are not often linked to conceptual or theoretical 

structures, anchored by a comprehensive search, oriented toward an 
extension of knowledge, or genera 1 izabl e . This state of affairs is often 
a matter of role constraints and planning. Schwen, Keller, et_ aX ( 1 974) 
developed a case study of a developfnent project that posed a number of 
questions about the installation of a Human Geography class. The case 
study served the professional needs of the client system but because a 
theoretical problefn was not posf^d nor new methodology developed, the 
criteria^ extension of knowledge^ linkage \^ith a theoretical structuret 
or general izabU 1ty were not met. The case could have been developed to 
meet these criteria. The case involved the mastery model and there were 
some innovative methodological patterns used* A report focusing on the 
theoretical understandings of the mastery model, modifications of mastery 
' methodology^ or both could have resulted in a case study worthy of 
consideration by an editorial board, 

Data based summaries of technological services, sumniaries of develop- 
ment projects, year-end reports, etc., are generally false exemplars. 
Many developrnental organizations, Audio-Visual Centers, etc, prepare 
sophisticated reports of their service activities. Most of this 
documentation would not be appropriate for consideration in AVCR. Although 
these reports obviously serve a useful function, they do not tend to 
address theoretical or conceptual questions, relate to the literature on 
educational technologyi extend theoretical knowledgei or permit general- 
izabHlty. Many adaptations could be devised to acconipllsh useful Inquiry. 
For exaniple, theoretical questions concerning the nature of service 
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organizations could be posed^ e.g., following Bennis (1953)i systematic 
comparisons or extensive enipirical case studies using the data froni 
service reports could be used to advance the precision or theory of 
educational technology organizations. Of course, all of the criteria 
would need to be applied to efforts of this kind. 

The current statement and restatements of devel opinent and message 
design models are for the most part false exemplarSi While these exercises 
may serve useful functions for 1 nstruct io..d 1 purposes, most discussions 
do not constitute an extension of knowledge or creative exploration, 
Also^ they are seldom integrated with conceptual or theoretical structures, 
d1sciplined^ or derived from comprehensive searches of literature. It 
is Interesting to note that even the first attempts to develop models 
were weak on the criterion of verification, There are fewi if any, 
published reports (including case studies) that have focused on explicit 
demonstration of a model as the central proijlem of the study. Appropriate 
Inquiry and publication in this formally unexplored area would be 
appropriate* Ellson's (1972) review of six studies demonstrating dramatic 
improvements in learning or efficiency outcomes was an Interesting example. 
He has recently assenibled twenty additional case studies (personal 
correspondence), e.g., Markle (1967) which, also, demonstrated unusual 
improvements as the result of systematic design process. This type of 
inquiry can be defended on the basis of the criteria. The Ellson review 
was disciplined I and the summary of case studies technique developed was 
effectively employed to develop g ener a 1 i zabl e and verifiable findings. 
There was a relationship developed with the traditional fields of 
scholarship. The exploration of the topic was reasonably comprehensive 
and the treatment and conclusions seemed to make intuitive good sense. 

The development of new techniques, methods of analysis, technological 
treatmentSi etc,, will be the subject of inquiry in educational technology 
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for many generations to come. For e;<ample. the recent interest in multi- 
dimensional scaling (Subkoviak 1975) could be the subject of inquiry 
across many different professional roles in Educational Technology. 
Film producers could use the tnethodologj' to analyze responses to films 
that elicit affective responses. Many film makers have been concerned 
about the incongruity between their intuitive reactions to •powerful' 
films and the absence of effects denion s trated with conventional methodologies 
such as Likert scales. The multidimensional scaling methodology, which 
has been used in marketing research for several years, has the advantage 
of utilizing non-metric data and sman numbers of subjects. It, also, 
seems quite sensitive to subjects' perceptions of concrete objects. In 
addition, the rnethodology could be used to assess client reactions to 
technological services, structural perceptions of cognitive content, etc. 
The point is that inquiry of this type, although primarny focused on 
increased precision in practice, could meet the previoisly stated 
criteria. 

Some of the criteria, when applied to precision oriented inquiry, 
should be weighted or treated differently. The criteria, verif 1 ab1 1 i ty , 
discipline, and creative exploration would be treated similarly in 
Inquiry directed to precision and in inquiry directed to understanding. 
However, the criteria, comprehensive search, general 1 zabi 1 1 ty^ integration 
with a conceptual structure, extension of knowledge, and sensibility may 
be applied to prediction oriented inquiry In a different fashion. For 
example, there may be a different emphasis on the criterion of a relation- 
ship to a conceptual or theoretical structure and comprehensive search. 
It would be important to link the inquiry to the appropriate methodolog- 
ical or closely related substantive literature. The inquiry may not 
necsssarily be linked to theoretical or conceptual understandings 1n 

18 



17 

education inquiry at large. This Is not to %hy that this is a desired 
state of affairs. It is, siniplys that predl ctive problems In a profession 
have a context or integrity that is closer to pragniatlc operations of day 
to day problems. The theory Is not being explored. An application of 
the theory is what may be new or novel. The search and linkage with the 
theoretical literature would tend to be more abbreviated and concentrated 
on methodological problems. Of course, professionals will need to be 
encouraged to reexamine their theoretical assumptions if Argyris' (1976) 
prophesy of theoretical stagnation 1s to be avoided. 

The criterion extension of knowledge follows logically from the 
discussion above, The ne\Nr knowledge in inquiry directed toward prediction 
will be of an applied nature. That is^ the professional commiinity will 
tend to add to its precision in practice by means of this type of inqiJiry, 
It should^ therefore^ be evaluated from this point of view by professionals 
who are expert in day to day -professional problefiis. 

The treatment of the remaining criteria, general i zabi 1 i ty ^ and 
intuitive good sense are quite Important and most distinct from 
application in inqutirj^ directed toward understanding. The Inquiry must 
make sense to the mature practicing technologists in the field before 
it should be published- It must make sense in respect to the body of 
Eubtle and intuitive understandings that accrue in professional practice 
over time. It niust make sense rn that it appears to offer the 
possibility of refining, improving, or modifying current practice. This 
criterion should not be used as an argument to retain the status quo* 
The criterion is advanced in conjunction with the assertion that mature, 
informal^ and intuitive understandings are as useful as formal criteria 
in judging scholarly activity of this type. 
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Finallyi the criterion of general i zabi 1 i ty must be applied with care. 
There may be times when no formal attempt to generalize would be in order. 
For example, a case study may deirions tra te a new technological process or 
application of a theoretical concept in such a dramatic fashion that It 
would bo worthy of consideration by an editorial boi^d. More typically, 
there will be complex trade offs between Internal validity and external 
validity to be considered. These trade offs have been discussed more 
completely in other sources (Snow, 1974), (Cook & CampbelU 1976), A few 
elaborations will be advanced here* In a profession with inquiry directed 
toward precision field studies will often be submitted. T: e inevitable 
threats to Internal validity must be carefully detailed by the author and 
weighed by the editorial boards* Incomplete discussions of the threats 
to internal validity would be reason for rejecting a publication. The 
issues of sampling and design will be perrenial problems wi.th the conse- 
quent compromises having implications for external validity. Of course, 
external validity will be of utmost concern in inquiry directed toward 
improved precision; quite often designs will be so constructed as to 
maximize external validity while investing "sufficient" efforts toward 
internal validity. With this emphasis on external validity and improved 
prediction the special topic of utility takes on added importance. The 
conventional practices of describing the probability of type I and type II 
errors in experimental designs and then moving on to inferences regarding 
the relationships between and among independent and dependent variables 
is, clearly, not sufficient. This could be appropriate for inquiry 
directed toward understanding theoretical relationships but in inquiry 
where increased prediction is the Intent, further analysis Is essential. 
It will be necessary for scholars to estimate the magnitude and scope of 
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their observed effects on ^jubsequent practice. For eKample^ Sego (1974) 
reported a significant apti tude-treatfnent interaction between col 1 ege 
students^ level of cognitive style and two types of programmed instruction 
materials. Sego extended the analysis of data by utilizing the Neyman- 
Johnson technique for estimating the region of the interaction* The 
analysis of the interaction indicated that less than two percent of the 
population would improve their learning by differential use of the sets 
of programmed instruction materials. In other words, it appeared as If 
only two percent of subsequent students using both sets of programmed 
instruction materials would do better than using either one of the sets 
alone. This analysis substantially altered the conclusions Sego advanced. 
He advised the professional community of scholars that his findings may 
suggest interesting and potentially useful theoretical relationships; 
however^ he also suggested that i./.med1ate improvements in learning would 
probably not be justified by the additional expense of developing and pro- 
ducing similar materials. The Sego example is the eKception rather than 
the rule in educational inquiry. More often than not significant ratios^ 
t-tests, multiple Rs, etc,, are reported without reference to the magni- 
tude of the relationships between independent and dependent variables. 
Conclusions are drawn without reference to the amount of variance that 
has been accounted for by the exper i men tors ' interventions. Our journals 
tend to be full of 'significant' but not important findings. 
Conclud ing R emarks 

This article has been primarily directed toward Judging inquiry that 
is subsumed by the scientific goal of prediction. The classic distinction 
between prediction in inquiry and inquiry directed toward increased 
understanding has been exaggerated so as to clarify the Issued of judging 
inquiry by practicing professionals. The distinction is one of emphasis 
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rather than exclusive orientation. At times practitioners could and should 
move to the laboratory to resolve practical and theoretical problems. In 
such cases their scholarly activities should be judged accordingly. 
Quite often, however, practitioners may be expected to engage in inquiry 
that will improve their precision in practice. This type of Inquiry 
should be judged the same criteria that would be used for other forms 
of inquiry. However, the application of the criteria may be weighted or 
adjusted to reflect the purpose of inquiry. 
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1* Is the inquiry publicly verifiable? 

2, Is the inquiry disciplined? 

3, Are the results of the inquiry general i zabl e? 

a. Is the Inquiry Internally valid or consistent? 

b. Is the inquiry externally valid or applicable? 

4, Is the inquiry integrated with a conceptual structure 
or theoretical framework? 

5, Is the inquiry directed toward an extension of knowledge? 

6, Is the inquiry derived from a comprehensive search? 

7, Does the inquiry demonstrate creative exploration? 

8, Does the inquiry make intuitive good sense? 



TABLE 1, A Summary of the Criteria for Judging Inquiry 1n 
Educational Technology* 
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